Retrospective analyses of large medical databases: what do they tell us?

The clinical study of human disease is complicated by interdependent variables, and powerful analytical tools are necessary to establish causal relationships. Prospective studies can be randomized and blinded to the investigators. These techniques protect prospective trials, somewhat, from the problems of bias in the study design and confounding by codependent variables. However, prospective clinical trials are difficult to perform, may require extended duration for adequate observations of human diseases, and are expensive to organize and perform. Retrospective studies may take less time and are less expensive than prospective studies because the data have already been measured. However, retrospective studies are susceptible to bias in data selection and analysis. Furthermore, confounding variables may go unrecognized because of inadequate knowledge of how they interrelate with the outcome of interest. Because of these limitations, retrospective data analysis may show associations among variables, but rarely establishes causal relationships. Dialysis has achieved widespread clinical acceptance because its efficacy is undisputed and the outcome without therapy is obvious. Patients with end-stage renal disease (ESRD) die of the complications of uremia, unless they are dialyzed or receive a renal transplant. Because the consequence of not dialyzing a patient with ESRD is so clear, dialysis was never subjected to the rigors of a prospective, randomized clinical trial. A lack of rigorous evaluation continued to characterize the development of renal replacement therapy for many years, although there were some exceptions, such as the National Cooperative Dialysis Study, which evaluated the effects of treatment time and urea removal on outcomes in hemodialysis (1). More often, retrospective, or a few prospective, clinical studies, usually in small numbers of patients, were all that were used to support changes in therapy. More recently, the recognition that dialysis patients in the United States have much higher mortality rates than patients in Europe and Japan (2) has led to a vigorous debate about the adequacy of dialysis practices in the United States. As part of this debate, there has been renewed interest in comparing clinical outcomes between different types of renal replacement therapy in order to determine if one therapy is superior to another. Because of the perceived urgency of the question, and because of the large number of patients and long time frames required to obtain sufficient statistical power in prospective, randomized clinical studies, many of these comparisons have been performed by retrospective observational analyses of large databases. The establishment of the United States Renal Data System (USRDS) has greatly facilitated retrospective analyses of outcomes in ESRD patients. The USRDS database contains patient-specific and center-specific data on essentially all ESRD patients treated in the United States, including demographic and medical information and ESRD treatment history (3). The USRDS presents summary statistics of these data annually and makes data files available to researchers who wish to test specific hypotheses. One example of an analysis of the USRDS database is the comparison between outcomes in peritoneal dialysis and hemodialysis reported by Vonesh and Moran in this issue of the Journal (4). Retrospective analyses of large databases, including the USRDS database, have sometimes produced contradictory, and even controversial, results. One example of these conflicting results is the purported association between dialyzer reuse and survival in hemodialysis patients. Based on an analysis of the USRDS database, Held et al. (5) and Feldman et al. (6) concluded that certain reuse practices were associated with an increased risk of mortality. However, a recent analysis of the same data, supplemented by data from other sources, calls into question this conclusion, and suggests that the apparent relationship between dialyzer reuse and mortality is confounded by other factors, such as dialysis therapy and anemia correction (7). The current study by Vonesh and Moran (4) is another example of conflicting results arising from different analyses of large ESRD databases. Although Vonesh and Moran show no difference in outcome between peritoneal dialysis and hemodialysis, other studies have shown hemodialysis to be associated with better (8) or worse (9) outcomes than peritoneal dialysis. The failure to obtain consistent findings when different investigators analyze the same, or a similar, database raises the question of the validity of such retrospective, observational studies. Because the results of these studies may influence national treatment practices, it is important to understand their limitations and clinical usefulness. Received December 15, 1998. Accepted December 16, 1998. Correspondence to Dr. Richard A. Ward, Kidney Disease Program, University of Louisville, 615 South Preston Street, Louisville, KY 40202-1718. Phone: 502-852-5757; Fax: 502-852-7643; E-mail: [email protected]